Introduction
The corpus I selected is a pool of music tracks i used for DJ sets previously, mainly include deep/ambient techno music. My main interest is to explore the relationship between the design and color themes of the music album covers and the content of the music itself. I chose this topic because I believe that the organic character of this music genre and inspiration from nature would often evoke particular imagery or emotions, which is tightly linked with colors. Additionally a few of the artists are also visual artists, so their album cover designs should strongly align with the music content.
I categorized the music tracks in four categories : Black, White, Blue and Pink, with 30 tracks in each category. When selecting songs, I also consider my own auditory perception, which is the idea of whether a song “gives me the vibe of this color.” From my own perspective, the songs with black color give me a sense of depth and mystery, reminding me of mountains, forests, and rocks at night. The songs in the blue playlist always remind me of the ocean, in the playlist I selected there are also songs with titles “the sea breathe” and “underwater acoustic.” The songs in the pink playlist are brighter and warmer, like the sunset on the horizon or cherry blossoms. The songs in the white playlist give a more ethereal feeling, connecting with an imagined world, such as the songs “transparent creature” and “River words.”
Can these subjective and imaginative perceptions be reflected and identified from quantifiable music features? This is the question that I am most interested in exploring. I hypothesize that bright and clear color themes on album covers would correspond to brighter music, while darker and more ambiguous cover themes may align with deeper and lower-frequency music. However, I am still uncertain about how other features such as rhythm and melody correspond to more abstract perceptions.
Overview of the audio features
Playlist features heatmap
I created a heatmap with the average value of the different audio features in different playlists. From this graph we can see that the corpus overall has a low valence level and high energy and danceability. The loudness value is been scaled from 0 to 1. The pink playlist has a higher valence and loudness compare to other playlists. The blue playlist has the highest energy value. And the black and white playlist have a relatively high daceability. As the corpus is electornic music, it has very high instrumentalness level and low acousticness level in all four playlists so these attributes are not invovled in this heatmap.
Valence-Energy graph
From this graph we can see that the pink category has a combination of sounds with high energy and low valence, or high valence with relatively low energy. The black category has high energy, with valence spread out the scale, but mainly at a low level. The blue category has medium - high energy with medium valence. The white category has medium energy and low valence.
Self Similarity Matrices
Pink
For the pink playlist I created the chroma and timbre self similarity matrix for two songs: Sakha-Infinity Remix and Cirrus Apparition. Overall the pink playlist shows clear musical structure in the self similarity matrix, as most of the matrices have dark checkerboard patterns. There is a main melody going throughout the song and some parts with some variations. For “Sakha” we can see 4 different parts in the track. The change of chroma and timbre are almost at the same time, and there is a fade out in the end. For “Cirrus Apparition”, the melody is repeating throughout the whole song, but from timbre there are two small segement at around 200s and 300s where variation happens. This repetitive rhythm give me a sense of a creature’s tendrils, gently meandering and ceaselessly delving into the unknown.
Blue
For the blue playlist I created the chroma and timbre self similarity matrix for Underwater Acoustics and Cosi Lontano. Overall I can see more difference in the structure of self similarity matrix for different tracks in this playlist.”Underwater Acoustics” shows a clear A-B-A-B structure. The color of the checkerboard is similar between the two parts because the ending has a very difference pattern compare to the other part of the music, which amplifies the scale of the similarity index on the plot. “Cosi lontano” shows variations in both chroma and timbre among the track. From the listener’s perspective, the song is still consistent because it has a steady percussion rhythm. The pad in this song changed several times, which is why there are many variations evident in the self-similarity matrix.
Black
For the black playlist I created the chroma and timbre self similarity matrix for Living nowhere with messy pigeons and Dagaz-Delusional circuits remix. Overall there are more smaller segments and the changes happens more rapidly in the black playlist compare to the others. “Living nowhere with messy pigeons” is an intro so there are no drum beats. We can see both chroma and timbre varies a lot throughout the track, and there are smaller musical segments shows from the checkerboard pattern. In “Dagaz-Delusional Circuits Remix”, we can see three major parts, but there are also always some smaller variations, represent by the straight light lines. These small music segments and rapid changes amplify the music’s sense of mystery and unpredictability.
white
In the similarity matrix for the white playlist, it shows similar patterns as the black playlist since there are also smaller music segments and a lot of small variations. But it is overall more consistent compare to the black one. In Synergy-Jupetto Mix there are many variations in chroma, because the melody was repeat playing with a shift in key. The timbre has more gradual changes throughout the track, as we can see a horizontal shift from black to yellow. This occurs because there are new instruments added on top of the preexisting melody. In River words, there are more consistency in chroma, but some small variations in timbre. This explain the music pattern of having the same melodic repeating throughout the track, but with some changes in sound texture. As the song is titled ‘River Words’, I can somehow see the gentle murmurs of a river represented by the main melody, with other sounds representing the wind, animals, and other plants in the forest.
Key and chord
Key histgram
I compared the distribution of keys within the four categories, where the darker color represents the major key and the lighter color represents the minor key. There are more major keys in pink and blue playlists compared to the black and white ones, which might contribute to a higher valence or positive emotional tone. In the pink playlist, there is a higher variation in keys that the tracks are evenly distributed across different keys. The black playlist on the other hand, shows a more concentrated distribution of keys, particularly centered around G major and B minor. In the white playlist C# is presented much more compare to the other playlists. Overall there are a lot of variety in the sharp and flat pitch classes in my corpus, but this cannot be directly associate with the complexity or the valence of the music.
Chordograms
I generated the chordograms for one of the most representative song track in each playlist. I applied euclidean distance metrics and manhattan norm. In the chordograms, there are more variations in the pink playlist. I displayed the one from the track Luna, where there is a repeat pattern of slight shifting downwards and shifting back in chords.As the song named “luna”, I associated these variations with the intermittently appearance and disappearance of moonlight. In Soave from the blue playlist there are also some variation in the middle that deviate from the main chord. The black (Descending) and white (Dwelling) playlist have a relatively more stable chord pattern that is consistent throughout the track.
Tempo
Tempo histgram
The tempo histograms reveal that the tempo across all four corpora tends to center around 120 bpm. However, the distribution of the tempo in the blue playlist appears to be more scattered compared to the others, indicating a greater variability in tempo within that playlist. In contrast, the pink and white playlist shows a very concentrated tempo distribution centered at around 125 bpm. As the corpus also included a few intro and outro tracks, it can explain the outliers with a tempo at around 100.
tempogram
I generated the tempogram for one song from each playlist. In the blue playlist there are a lot more variations in tempo compared to the other playlist. I plot the tempo graph for the song The Sea Breathe. It shows an average slow tempo with a fade in and a fade out. The tempo depicted on the tempogram s not very ascertained because there are no clear drum beats in this track. The slow tempo gives me a sense of relaxation, like sitting by the seaside, feeling the gentle sea breeze, and watching the flow of the waves.For the pink playlist I created the tempogram for Ofrenda - Alfred Czital Remix, it has a much higher tempo at around 135, and the tempo is very clear and consistent. With black playlist (Court of light) and white play list (Transparent creatures) it shows a similar pattern with an average tempo around 125.
Classification
K-Nearest Neighbors (KNN) classifier
I applied a K-Nearest Neighbors (KNN) classifier to distinguish the four playlists with different music album color. From the result we can see that the classifier is relatively more effective at identifying music in a white color album, and it has very low performance when identifying blue album. However the overall accuracy is only 20%, which means the probability of accurate prediction is lower than chance. It implies that either there is no clear difference in music with different color albums, or the differences can’t be identified from spotify api features.
CART Decision Tree
By using CART Decision tree for classification task, the accuracy
raised to 37.5%.
This classifier is very good at identify black playlists.
The most important features identified by the classifier are: mode_key,
energy, valence and tempo. However since the sample size is very small,
the classifier might be over fitting the sample data instead of
detecting the general pattern to identify color from audio features.
Conclusion
1. Brighter hues, brighter melodies
Overall, this study explored the relation in musical features and album cover color in the genre of ambient techno. The music with a pink album cover shows higher valence level and loudness, while blue color songs shows low valence but higher energy level. Black and white color music have higher dancebility compare to the lighter color album.
2. Color and musical structures
The self similarity matrices show that the pink color music have a clearer sturcture in the change of timbre and chroma, while the black and white color music have more rapid change and smaller musical segments. The blue album music have different type of musical structure in different tracks.
3. Color and variations
By delving deeper into the key and tempo of the music. We can see that pink and blue album cover music shows more variations in audio features compare to the albums with black and white cover. The blue playlist shows more variation in tempo, and the pink playlist shows more variation in key and chords.
4. Can the “Mindless Machine” tell the color of music?
Unfortunately, the two types of classifier I applied all can’t achieve high accuracy in predicting the album cover color from the music features, which means it is hard to identify the differences in color from quantifiable features. Additionally, the selection of colors for music album covers is subjective, as individuals associate colors with music in their own unique ways. This is also influenced by their social and cultural background. Therefore, the low accuracy of mathematical models was expected.
5. Conclution, limitation and next step
This study discovered various patterns in music with different album cover color, which could be potentially beneficial for researchers interested in the relation between music and color. However, since many abstract auditory experiences largely depend on timbre, which is challenging to quantify and interpret, this study did not focus extensively on this aspect, making it difficult to translate imagery concepts into quantifiable music features. For the next step, I should focus more on understanding the psychological mechanisms underlying color-music associations and develop more robust models to exploring the intricate relationship between color and music.